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Abstract. Many theorems about Kolmogorov complexity rely on exis- 
tence of combinatorial objects with specific properties. Usually the prob- 
abilistic method gives such objects with better parameters than explicit 
constructions do. But the probabilistic method does not give "effective" 
variants of such theorems, i.e. variants for resource-bounded Kolmogorov 
complexity. We show that a "naive derandomization" approach of re- 
placing these objects by the output of Nisan-Wigderson pseudo-random 
generator may give polynomial-space variants of such theorems. 
Specifically, we improve the preceding polynomial-space analogue of Much- 
nik's conditional complexity theorem. I.e., for all a and 6 there exists a 
program p of least possible length that transforms a to & and is simple 
conditional on b. Here all programs work in polynomial space and all 
complexities are measured with logarithmic accuracy instead of polylog- 
arithmic one in the previous work. 

1 Introduction 

Many statements about Kolmogorov complexity may be proven by applying 
some combinatorial constructions like expanders or extractors. Usually these 
objects are characterized by some parameters, and one may say which parame- 
ters are "better" . Very often the probabilistic method allows one to obtain these 
objects with much better parameters than explicit constructions do. But exploit- 
ing the probabilistic method causes exponential-space brute-force search for an 
object satisfying the necessary property. And if this search is performed while de- 
scribing some string to obtain an upper bound on its complexity then this bound 
cannot be repeated for a resource-bounded version of complexity, even for the 
polynomial-space one. On the other hand, replacing the probabilistic method by 
an explicit construction weakens the statement due to worse parameters. 

We present a technique that combines advantages of both probabilistic and 
explicit construction methods. The key idea is to substitute a random object 
with a pseudo-random one still possessing the necessary property. The employed 
property of a pseudo-random generator is the indistinguishability one: its output 
cannot be distinguished from a random string by boolean circuits of constant 
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depth and polynomial size. The Nisan-Wigderson generator satisfies this condi- 
tion. If the necessary property can be tested by such circuits then it holds for 
a pseudo-random object with approximately the same probability as for a truly 
random object, so a search among all seeds may be perormed. Unfortunately, it 
is not clear how to build such a circuit for the extractor property and similar 
ones, so we relax the property in a way that allows both proving the theorem 
and building polynomial constant-depth circuits. 

This "naive derandomization" idea has been recently applied by Andrei Ro- 
mashchenko ([18 ) in another situation: a probabilistic bit-probe scheme with 
one-sided error for the membership problem is constructed. 

By exploiting the new method we improve the previous result |13| generaliz- 
ing Muchnik's conditional complexity theorem. The original theorem !9] states 
that for all a and b of length n there exists a program p of length C(a|fe)+0(log n) 
that transforms b to a and has complexity O(logn) conditional on a. In |13j this 
result is restated for space-bounded complexity with gap rised from O(logn) to 
0(log 3 n). The main idea was to employ a property of extractors, proven in [2]: 
in an extractor graph in any sufficiently big subset S of the left part there are 
few vertices with all right-part neighbours having indegree from S twice greater 
than average. We refer to this property as to "low-congesting" one. Best ex- 
plicit extractor constructions yield a space-bounded version of the theorem with 
polylogarithmic precision. In this paper we replace an explicit extractor by a 
pseudo-random graph, that does not necessary have the extractor property, but 
enjoys the low-congesting property for "relevant" subsets S. This replacement 
leads to decreasing the precision back to logarithmic term. 

The rest of the paper is organized as follows. In Sect. [2j we give formal def- 
initions of all involved objects and formulate necessary results. In Sect. [3] we 
formally state our space-bounded variant of Muchnik's theorem and specify all 
details of the proof. 

2 Preliminaries 

2.1 Kolmogorov complexity 

Let V be a two-argument Turing machine. We refer to the first argument as 
to the "program" and to the second argument as to the "argument" . (Plain) 
Kolmogorov complexity of a string x conditional on y with respect to V is the 
length of a minimal program p that transforms y to x, i.e. 

Cv(x I y) = min{p: V(p,y) = x} 

There exists an optimal machine IA that gives the least complexity up to an 
additive term. Specifically, VV 3c Vx, y Cu{x\y) < C\>(x\y) + c. We employ such 
a machine U, drop the subscript and formulate all theorems up to a constant 
additive term. The unconditional complexity G(x) is the complexity with empty 
condition C(x | e), or the length of a shortest program producing x. 



Now we define the notion of resource-bounded Kolmogorov complexity. Roughly 
speaking, it is the length of a minimal program that transforms y to x effi- 
ciently. Formally, Kolmogorov complexity of a string x conditional on y in time 
t and space s with respect to V is the length of a shortest program p such that 
y(p> V) = x i an d the computation of V(p, y) works in t steps and uses s cells of 
memory. This complexity is denoted by C\) s (x \ y). Here the choice of V alters 
not only complexity, but also time and space bounds. However, there still exists 
an optimal machine in the following sense: 

Theorem 1. There exists a machine U such that for any machine V there ex- 
ists a constant c such that for all x, y, s and t it is true that Cy (x \ y) < 
C c v s ' cn ° st (x\y)+c. 

This paper deals with space bounds only, so the time-bound superscript in all 
notations is dropped. Also, the machine subscript is dropped as before and all 
theorems are formulated with a constant additive term in all complexities and a 
constant multiplicative term in all space bounds. 

2.2 Extractors 

An extractor is a function that extracts randomness from weak random sources. 
A fc-weak random source of length n is a probabilistic distribution on {0, 1}™ 
with minentropy greater than k. The last condition means that no particular 
string occurs with probability greater than 2 . An extractor with parameters 
n, m, d, k, e is a function Ext: {0, 1}™ x {0, l} d -> {0, l} m , such that for any 
independent fc-weak random source x of length n and uniform distribution u on 
{0, l} d the induced distribution Ext(x, u) on {0, 1}™ is e-close to uniform, that 
is, for any set Y C {0, l} m its probability differs from its fraction by at most e. 
This is interpreted as follows: an extractor receives n weakly random bits and d 
truly random bits independent from the first argument and outputs m almost 
random bits. 

Like any two-argument function, an extractor may be viewed as a bipartite 
(multi-)graph: the first argument indexes a vertex in the left part, the second 
argument indexes an edge going from this vertex, and the value indexes a vertex 
in the right part which this edge directs to. That is, the graph has N = 2™ 
vertices on the left, M — 2™ vertices on the right, and all left-part vertices 
have degree D = 2 d . Throughout the paper, we say that a bipartite graph 
has parameters (n, m, d) if the same holds for it. For the sake of clarity we 
usually omit these parameters in extractor specifications. The extractor property 
may be also formulated in terms of graphs: for any left-part subset S of size 
greater than K — 2 k and for any right-part subset Y the fraction of edges 
directing from S to Y among all edges directing from S differs from the fraction 
of vertices from Y among all right-part vertices by at most e. Put formally, 
\\Y\/M - E(S,Y)/E{S,M)\ < e, where E(S, T) is the number of edges diredting 
from S to T and M slightly abusively denotes both the right part and the number 
of vertices in it. A proof of equivalence may be found in [16] ■ 



It is proven by the probabilistic method (see, for example, |16j ) that for all n, 
k and e there exists an extractor with parameters d — log(n — k) + 2 log(l/e) + 
0(1) and m = k + d — 21og(l/e) — 0(1). Nevertheless, no explicit (that is, 
running in polynomial time) construction of such an extractor is known. Best 
current results ([IT], [50]) for m = k use d — polylogn truly random bits. A 
number of explicit extractors including those in [7J, [4] and [5] use O(logn) truly 
random bits but output only (1 — a)k almost random bits. Such constructions 
are insufficient for our goals. 



2.3 Nisan-Wigderson generators 

The Nisan-Wigderson pseudo-random generator is a deterministic polynomal- 
timc function that generates n pseudorandom bits from poly log (n) truly random 
bits. The output of such a generator cannot be distinguished from a truly random 
string by small circuits. Specifically, we exploit the following theorem from |15j . 
(The statement was initially proven by Nisan in paper |14j). 

Theorem 2. For any constant d there exists a family of functions G n : {0, l} k — > 
{0, l} n ; where k = 0(log 2d+e n), such that two properties hold: 

Computability: G is computable in workspace poly (A:); 

Indistinguishability: For any family of circuits C n of size poly(n) and depth 
d for any positive polynomial p for all large enough n it holds that: 

\Pmb x {C n {G n {x)) = 1} - Prob y {C n (y) = 1}| < -L 

p[n) 

where x is distributed uniformly in {0, l} k and y — in {0, 1}". 

By rescaling the parameters we get the following 

Corollary 1. For any constant d there exists a family of functions G n : {0, l} fe — > 
{0, 1} N , where k = poly(n) and N = 2°^ n \ such that two properties hold: 

— G is computable in workspace poly(n); 

— For any family of circuits C n of size 2°("' and depth d, for any constant c 
and for all large enough n it holds that: 

\Prob x {C n (G n (x)) = 1} - Prob y {C n (y) = 1}| < T cn . 

Here all constants in poly- and O-notations may depend on d but do not depend 
on k, n and C n . 

The last corollary implies the following basic principle: 

Lemma 1. LetC n be some set of combinatorial objects encoded by boolean strings 
of length 2° LetV be some property satisfied for fraction at least a fraction of 
objects in C n that can be tested by a family of circuits of size 2°^ and constant 
depth. Then for sufficiently large n the property V is satisfied for a fraction at 
least a/2 of values of G n , where G n is the function from the previous corollary. 



2.4 Constant-depth circuits for approximate counting 

It is well-known that constant-depth circuits cannot compute the majority func- 
tion. Moreover, they cannot compute a general threshold function that equals 1 
if and only if the fraction of l's in its input exceeds some threshold a. Neverthe- 
less, one can build such circuits that compute threshold functions approximately. 
Namely, the following theorem holds: 

Theorem 3 (|lj). Let a G (0,1). Then for any (constant) e there exists a 
constant- depth and polynomial- size circuit C such that C{x) — if the fraction 
of l's in x is less than a — e and C(x) = 1 if the fraction of l's in x is greater 
than a + e. 

Note that nothing is promised if the fraction of l's is between a — e and a + e. 
So, the fact that C(s) = guarantees only that the fraction of l's is at most 
a + e, and C(s) = 1 — that it is at least a — e. 

3 Muchnik's theorem 
3.1 Subject overview 

An. Muchnik's theorem [9] on conditional Kolmogorov complexity states that: 

Theorem 4. Let a and b be two binary strings such that C(a) < n and C(a\b) < 
k. Then there exists a string p such that 
. C(a\p,b) = 0(logn); 

• C(p) < fc + 0(logn); 

• C(p\a) = O(logn). 

The constants hidden in O(logn) do not depend on n,k,a,b,p. 

Informally, this theorem says that there exists a program p that transforms 
b to a, has the minimal possible complexity C(a\b) (up to a logarithmic term) 
and, moreover, can be easily obtained from a. (The last requirement is crucial, 
otherwise the statement trivially reformulates the definition of conditional Kol- 
mogorov complexity.) 

Several proofs of this theorem are known. All of them rely on the existence of 
some combinatorial objects. The original proof in [9] leans upon the existence of 
bipartite graphs with specific expander-like property. Two proofs by Musatov, 
Romashchenko and Shen [13] use extractors and graphs allowing large on-line 
matchings. Explicit constructions of extractors provide variants of Muchnik's 
theorem for resource-bounded Kolmogorov complexity. Specifically, the following 
theorem is proven in |13j : 

Theorem 5. Let a and b be binary strings of length n, and k and s be integers 
such that C s (a|6) < k. Then there exists a binary string p, such that 



• C°( s)+poly(n) (a\p,b) =(9(log 3 n); 

• C S ( P ) <k + 0(\ogn); 



C poly(n) (p|a) = 0(log 3 n), 



where all constants in O- and poly -notations depend only on the choice of the 
optimal description method. 

One cannot reduce residuals to O(logn) since all explicit extractors with such 
seed length output only (1 — a)k bits, but p is taken as an output of an ex- 
tractor and should have length k. However, an application of our derandomiza- 
tion method will decrease conditional complexities from 0(log 3 n) to O(logn) + 
O(loglog s) at the price of increasing the space limit in the last complexity from 
poly(n) to O(s) + poly(n). If s is polynomial in n then all space limits become 
also polynomial and all complexity discrepancies become logarithmic. 



3.2 Proof overview 



Before we proceed with the detailed proof, let us present its high-level descrip- 
tion. The main idea is the same as in all known proofs: p is a fingerprint (or 
hash value) for a constructed in some specific way. This fingerprint is chosen via 
some underlying bipartite graph. Its left part is treated as the set of all strings of 
length n (i.e., all possible a's) and its right part is treated as the set of all possible 
fingerprints. To satisfy the last condition each left-part vertex should have small 
outdegree, since in that case the fingerprint is described by its ordinal number 
among a's neighbours. To satisfy the first condition each fingerprint should have 
small indegree from the strings that have low complexity conditional on b (for 
arbitrary 6). 

If resources are unbounded then the existence of a graph satisfying all con- 
ditions may be proven by the probabilistic method and the graph itself may be 
found by brute-force search. In the resource-bounded case we suggest to replace 
a random graph by a pseudo-random one and prove that it still does the job. 
The proof proceeds in several steps. Firstly, in Sect. |3.3] we define the essential 
graph property needed to our proof. We call this property low-congestion. It 
follows from the extractor property, but not vice versa. Secondly, in Sect.|3.4|we 



specify the instrumental notion of space-bounded enumerability and prove some 
lemmas about it in connection to low-congesting graphs. Next, in Sect. [375| we 
employ these lemmas to prove that the low- congesting property is testable by 
small circuits. Hence, by applying the main principle (lemma [I]) this property is 
satisfied for pseudo-random graphs produced by the NW generator as well as for 
truly random ones. Moreover, a seed producing a graph with this property may 
be found in polynomial space. Finally, in Sect. |3.6| we formulate our version of 
Muchnik's theorem and prove it using the graph obtained on the previous step. 
I.e., we describe the procedure of choosing a fingerprint in this graph and then 
calculate all complexities and space requirements and assure that they do not 
exceed their respective limitations. 



3.3 Low-congesting graphs 

In fact, the proof in |13| does not use the extractor property, but employs only 
its corollary. In this section we accurately define this corollary in a way that 
allows derandomization. 

Fix some bipartite graph with parameters (n, m, d) and an integer k < n. 
Let there be a system S of subsets of its left part with the following condition: 
each Se5 contains less than 2 fc vertices, and the whole system S contains 2°( n ) 
sets (note that there are J2i=i ^2™ > 2°( n ) different sets of required size, so the 
last limitation is not trivial). We refer to such systems as to "relevant" ones. 
Having fixed a system S, let us call the sets in it relevant also. 

Lemma 2. Let S k = {S C {0, 1}" | 3b3s \b\ = n and S = {x \ C s {x\b) < k}}. 
Then the system Sk is relevant. 

Proof. By a standard counting argument, each set S £ Sk contains less than 2 fe 
elements. If b is fixed then the described sets are expanding while s is rising. 
Since the largest set is smaller than 2 fe , there are less than 2 k different sets for 
a fixed b. Since there are 2™ different strings b, the total size of Sk is bounded 
by 2"2 fc = 2°W. 

Considering a modification of the upper system Sk,s — {S C {0,1}™ | 3b3s < 
s S = {x | C s (x\b) < k}}, one may note that it is relevant as well. 

Now fix some relevant system S and take an arbitrary set S in it. Call the a- 
clot for S the set of right-part vertices that have more than aDK/M neighbours 
in S (that is, at least a times more than on average). Call a vertex x £ S 
a- congested (for S) if all its neighbours lie in the a-clot for S. Say that G 
is (a, ft) -low- congesting if there are less than j3K a-congested vertices in any 
relevant S. 

Following [2], we prove the next lemma: 

Lemma 3. Let G be an extractor graph with parameters n, m, d, k, e. Then 
for any a > 1 the graph G is (a, -^je) -low- congesting. 

Proof. In fact in an extractor graph there are less than -^zjeK a-congested 
vertices in any set S of size K, not only in relevant ones. We may treat S as 
an arbitrary set of size exactly K: since a congested vertex in a subset is also a 
congested vertex in the set, an upper bound for the number of congested vertices 
in the set holds also for a subset. 

Let Y be the a-clot for S, and \Y\ = SM. Then the fraction \Y\/M of vertices 
in Y equals S and the fraction E(S,Y)/E(S, M) of edges directing from S to 
Y is greater than ad (by the definition of clot). A standard counting argument 
implies only S < ±, but by the extractor property E(S, Y)/E(S, M)-\Y\/M < e, 
so (a — 1)8 < e, i.e. S < -£zi £ - Next, let T C S be the set of a-congested 
vertices in S, and \T\ = (3K. All edges from T direct to vertices in Y, so at least 
D\T\ — (3DK edges direct from S to Y. In other words, the fraction of edges from 
S to Y is at least (3. By the extractor property it must differ from the fraction 
of vertices in Y (that equals S) by at most e. So, (3 < S + e < -£zi£ + £ = 



Putting it all together, the number of a-congested vertices in any relevant S is 
less than ^^eK, so the graph is (a, ^ye)-low-congesting, q.e.d. 

3.4 Space-bounded enumerability 

Say that a system S is enumerable in space q if there exists an algorithm with 
two inputs i and j working in space q that either outputs the jih element of the 
zth set from S or indicates that at least one of the inputs falls out of range. Note 
that for a polynomial space bound enumeration is equivalent to recognition: 
if one may enumerate a set then one may recognize whether a given element 
belongs to it by sequentially comparing it to all enumerated strings, and if one 
may recognize membership to a set then one may enumerate it by trying all 
possible strings and including only those accepted by the recognition algorithm. 
Only small auxiliary space is needed to perform these modifications. 

Lemma 4. The system <S fc g = {S C {0, 1}™ | 363s < s \b\ = n and S = {x \ 
C s (x\b) < k}} is enumerable in space 0(s) + poly(n). 

Proof. Assume firstly that a set S is given (by specifying b and s < s) and 
show how to enumerate it. Look through all programs shorter than k and launch 
them on b limiting the space to s and counting the number of steps. If this 
number exceeds c s (for some constant c depending only on the computational 
model) then the current program has looped. If the looping is detected or if the 
program exceeds the space limit it is terminated and the next one is launched. 
Otherwise, if the program produces an output, then a check whether it has not 
been produced by any previous program is performed. This check proceeds as 
follows: store the result, repeat the same procedure for all previous programs 
and compare their results with the stored one. If no result coincides then include 
the stored result in the enumeration, otherwise skip it and in both cases launch 
the next program. Emulation of a program requires O(s) space and no two 
emulations should run in parallel. All intermediate results are polynomial in n, 
so a total space limit 0(s) + poly(n) holds. 

Specifying a set S by b and s is not reasonable because a lot of values of s 
may lead to the same set. (And if s = 2 W (") then the number of possible indexes 
(b, s) exceeds the limit 2°( n ) on the size of S). Instead, call a limit s pivotal if 
{x | C s (x\b) < k} 7^ {x | C s ~ (x\b) < k} and consider only pivotal limits in the 
definition of <Sfc j5 . Clearly, the latter modification of definition does not affect 
the system itself. The advantage is that there are less than 2 k pivotal limits, and 
the iih pivotal limit Sj may be found algorithmically in space O(si) + poly(n). 
Indeed, it is sufficient to construct an algorithm recognizing whether a given 
limit is pivotal, and the latter is done by a procedure similar to one described 
in the first paragraph: try all possible programs in space s, and in case they 
produce an output, check whether it is produced by any program in space s — 1. 
If at least one result is new then s is pivotal, otherwise it is not. Putting all 
together, a set S in Sk,s is defined by a word b and the ordinal number i of a 
pivotal space limit Sj. Knowing these parameters, one may enumerate it in space 



0(s) + poly(n). Only 0(n) additional space is needed to look over all possible 
values of b and i, so the declared bound is meeted. 

The next lemma indicates that if a system S is enumerable in small space, 
then the same holds for the system of congested subsets of its members. We call 
a bipartite graph computable in space q if there exists an algorithm working in 
space q that receives an index of a left-part vertex and an index of an incident 
edge and outputs the right-part vertex this edge directs to. 

Lemma 5. Let S be a system of relevant sets enumerable in space s, G be a 
bipartite graph computable in space q and a > 1 be a (rational) number. Then the 
system Con Q S = {T \ 3S € S for which T is the set of a- congested vertices} is 
computable in space 0(max{s, g}) +poly(n). Moreover, the iteration (Con a ) r <S 
is also computable in space 0(max{s, g}) + poly(n) with constants in O- and 
poly- notations not depending on r, but possibly depending on a. 

Proof. Since for space complexity enumeration and recognition are equivalent, it 
is sufficient to recognize that a vertex x is a-congested. The recognizing algorithm 
works as follows: having a set S and a vertex x £ S fixed, search through all 
neighbours of x (this requires space O(q) + poly(n)) and for each neighbour 
check whether it lies in the a-clot for S. If all neighbours do then x is congested, 
otherwise it is not. Having a neighbour y fixed, the check is performed in the 
following way: enumerate all members of S (using O(s) + poly(n) space), for 
each member search through all its neighbours (using O(q) +poly(n) space) and 
count the number of these neighbours coinciding with y. Finally, compare this 
number with the threshold aDK/M. Note that no two computations requiring 
space s or q run in parallel and all intermediate results need only polynomial 
space, so the total space requirement is 0(max{s, g}) + poly(n), as claimed. Note 
also that O- notation is used only due to the possibility of computational model 
change, not because of the necessity of looping control. If a computational model 
is fixed then the required space is just max{s, q} + poly(n). 

The last observation is crucial for the "moreover" part of lemma. Indeed, by 
a simple counting argument the fraction of a-congested vertices in S is at most 
1/a. That is why after at most log Q 2" = 0(n) iterations the set of congested 
variables becomes empty. Each iteration adds poly(n) to the space requirement, 
so the overall demand is still max{s, q} + poly(n) (with greater polynomial), as 
claimed. 

3.5 Derandomization 

In this section we show that, firstly, the low-congesting property may be (approx- 
imately) recognized by 2°(™)-sized constant-depth circuits, secondly, that there 
are low-congesting graphs in the output of Nisan-Wigderson pseudo-random gen- 
erator and, thirdly, that one can recognize in polynomial space whether the 
NW-generator produces a low-congesting graph on a given seed. Put together, 
the last two lemmas provide a polynomial-space algorithm outputting a seed on 
which the NW-generator produces a low-congested graph. 



Let us encode a bipartite graph with parameters (n, m, d) by a list of edges. 
The length of this list is 2 n 2 d m: for each of 2™ left-part vertices we specify 2 d 
neighbours, each being m-bit long. 

Lemma 6. Let Q be the set of all bipartite graphs with parameters (n, m, d) 
encoded as described above. Let k be an integer such that 1 < k < n, and let e be 
a real positive number. Then there exists a circuit C of size 2°(") and constant 
depth defined on Q , such that: 

— If G is a (k, e)-extractor then C(G) — 1; 

— If C(G) — 1 then G is a (2.01, 2.0ls) -low-congesting graph for Sk from 
lemma\M 

Proof. We build a non-uniform circuit, so we may assume that Sk is given. 
We construct a single circuit approximately counting the number of congested 
vertices in a given set, then replicate it for each relevant set and take conjunction. 
Since there are less than 2 n+k relevant sets, this operation keeps the size of circuit 
being 2°( n > . We proceed by constructing a circuit for a given set S. A sketch of 
this circuit is presented on Fig. |3.5| 

The size of the input is \S\ ■ 2 d m. We think of it as of being divided into 
\S\ blocks of 2 d segments of length m. We index all blocks by elements x G S 
and index all segments of the block x by vertices y incident to x. It is easy 
to see that there is a constant-depth circuit that compares two segments (that 
is, has 2m inputs and outputs 1 if and only if the first half of inputs coincides 
with the second half). On the first stage we apply this circuit to every pair 
of segments, obtaining a long 0-1-sequence. On the second stage we employ a 
counting circuit with IS]!) — 1 arguments that is guaranteed to output 1 if more 
than 2. 01D K /M of its arguments are l's and to output if the number of l's is 
less than 2DK/M. By theorem [3] there exists such a circuit of polynomial (in the 
number of arguments) size and constant depth. For all segments y a copy of this 
circuit is applied to the results of the comparison of y to all other segments. If y 
lies in the 2.01-clot then the respective copy outputs 1, and if it outputs 1, then y 
lies in 2-clot. On the third stage we take a conjuction of all second-stage results 
for the segments lying in the same block x. If this conjunction equals 1 then 
all images of x lie in 2-clot, that is, x is 2-congested. Conversely, if x is 2.01- 
congested then the conjunction equals 1. Finally, we utilize another counting 
circuit with \S\ inputs that outputs if more than 2.0l£-fT of its inputs are l's 
and outputs 1 if less than 2eK of its inputs are l's. This circuit is applied to all 
outputs of the third stage. If the final result is 1 then less than 2.QleK elements 
of S are 2.01-congested; and if less than 2eK elements of S are 2-congested then 
the final result is 1. 

The last claim holds for any relevant S. On the very last stage we take a 
conjunction of results for all S. If this conjunction is 1 then less than 2.01eK 
elements in each S are 2.01-congested, and if G is a (fc, e)-extractor then by 
lemma [3] less than 2eK elements in each S are 2-congested and hence this con- 
junction equals 1. 

Since there are at most 2°'™' gates on every stage and each stage has constant 
depth, the overall circuit has also 2°^ gates and constant depth, as claimed. 
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Fig. 1. A sketch of the circuit for fixed S. 



Lemma 7. Let n, m — k, d = O(logn) and e be such parameters that a random 
bipartite graph with parameters (n, d, m) is a (k, e)-extractor with constant 
probability p > 0. Let q be the depth of the circuit from the previous lemma. Let 
NW: {O.I} 1 -> {0,1}^, where I = 0(log 29+6 ) and N = 2 n 2 d m, be the Msan- 
Wigderson generator from corollary^ Then Prob M {C(./VW(u)) = 1} > | for 
sufficiently large n, where C is the circuit from the previous lemma. 

Proof. This is a straightforward application of lemma [T] By the previous lemma 
if a graph G is an extractor then C(G) = 1, so Prob(j{C(G) = 1} > p. Since C 
is a constant-depth circuit, the property C(G) = 1 is tautologically testable by 
a constant-depth circuit. By lemma [T]Prob u {C(A r W / (u)) = 1} > |, q.e.d. 

Consider the following problem 1Z: find a string u € {0, 1}' such that the 
graph NW(u) is (2.01, 2.0l£-RT)-low-congesting with respect to Sk. yS - 

Lemma 8. The problem TZ is solvable in space 0(s) + poly(n). 

Proof. The existence of a solution follows from the previous lemma. Since we 
care only about the space limit, the search problem may be replaced by the cor- 
responging recognition problem. The space bound for the latter one arises from 



corollary [IJ lemma [4] and lemma [5j Indeed, by corollary [l] the graph NW(u) is 
computable in polynomial space, and by lemma|4]the system Sk , s is enumerable 
in space O(s) + poly(n). Hence by lemma[5]the system C0n2.cu is also enu- 
merable in space 0(s) + poly(n), therefore one may easily check whether each set 
in Con2.oi<5fc jS contains less than 2.QleK elements, thus solving the recognition 
analogue of TZ. Only polynomial extra space is added on the last step. 

3.6 Proof of the theorem 

Now we proceed by formulating and proving our version of Muchnik's theorem. 

Theorem 6. Let a and b be binary strings of length less than n, and s and k be 
numbers such that C s (a|6) < k. Then there exists a binary string p, such that 

• C° (s)+poly(Tl) (a|p,6) = 0(log logs + log n); 

• C s (p) <k + 0(\ogn); 

• C° (s)+poly(,l) (p|a) = 0(log logs -flog n), 

where all constants in O- and poly -notations depend only on the choice of the 
optimal description method. 

Proof. Basically the proof proceeds as the respective proof of theorem [5] in [13] . 
Here we replace an explicitly constructed extractor by a pseudorandom graph 
described in Sect. 13.51 

Let e be a small constant and let d — 0(log n) be such that a random bipartite 
graph with parameters (n, m = k, d) is a (fc, e)-extractor with probability greater 
than some positive constant /j,. Let I be a parameter and NW be a function 
from lemma [T] By lemmas [7] and [6] the output of NW is a (2.01, 2.01e-?T)-low- 
congesting graph for Sk with probability at least fJ-/2. Applying the program 
from lemma[8j one may find in O(s) + poly(n) space a seed u for which NW{u) 
is a (2.01, 2.0l£i4T)-low-congesting graph for Sk, s - This u has low complexity: to 
perform this search one needs to know parameters n, k and I — poly(n), that is, 
O(logn) bits, and the space bound s, that requires logs bits. The last number 
may be reduced to log logs, because the space bound s may be replaced by the 
least power of 2 exceeding s keeping the needed space to be 0(s) + poly(n). For 
what follows, fix this seed u and the graph G = NW(u). 

By definition of the low-congesting property the number of 2.01-congested 
vertices in the set S — {x | C s (x\b) < k} does not exceed 2.01eK. Clearly, 
a belongs to S. Firstly suppose that it is not 2.01-congested. Then it has a 
neighbour outside the 2.01-clot for S. This neighbour may be taken as p. Indeed, 
the length of p equals m — k, hence its complexity is also less than k + O(l). 
To specify p knowing a, one needs to construct the graph NW(u), for which 
only O(logn) + O (log logs) bits are necessary, and to know the number of p 
among a's neighbours, which is at most d = O(logn). Finding a given b and p 
proceeds as follows: construct G, enumerate S and choose the specified preimage 
of p in S. Then a is determined by the information needed to construct G 
(0(logn + log logs) bits), information needed to enumerate S (that is, k, logs 



and b) and the number of a among preimages of p (since p is not in the 2.01-clot, 
there arc not more than 2.01DK/M — 2.011? preimages, so 0(d) — O(logn) bits 
are required). Summarizing, we get O (log n)+0 (log log s) bits and O(s) +poly(n) 
space needed. Note that the fact that m — k is crucial here: in the case m = 
(1 — a)k the number of required bits would increase by ak and exceed the bound. 

Now we turn to the case where a is 2.01-congested. By lemma [5] the set 
C0n2.cu S of 2.01-congested vertices is enumerable in O(s) +poly(rt) space. Take 
parameters n\ =n,mi=kx = logifi = log(2.01e.K"), d\ = O(logn) and S\—e 
such that an extractor with these parameters exists with probability greater 
than /i. Lemmas [6j [7] and [8] are applicable to the new situation, so we may find a 
new u\ such that the graph G\ — NW(ui) is (2.01, 2.01eifi)-low-congesting for 
Con2.oi<5fc !S with probability at least /z/2. By assumption, a belongs to the set 
Con 2 .oi S. If it is not 2.01-congested in the new set then we choose p analogously 
to the initial situation. Otherwise we reduce the parameters again and take a 
low-congesting graph for Con?, 01 Sk, s , and so on. By the "moreover" part of 
lemma [5] this reduction may be performed iteratively for arbitrary number of 
times keeping the space limit to be O(s) + poly(n). 

The total number of iterations is less than \og 1 / 2 oie ^ = O(logn). Finally p 
is defined as a neighbour of a not lying in the 2.0Tclot in graph G^. To find p 
knowing a one needs to know i and the same information as on the upper level. 
To find a knowing p and b also only specifying i is needed besides what has been 
specified on the upper level. So, all complexities and space limits are as claimed 
and the theorem is proven. 
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